Skip to content

Document category evidence-sharing experiment - #52

Merged
Stanback merged 1 commit into
mainfrom
research/category-sharing-experiment
Oct 26, 2026
Merged

Stanback merged 1 commit into
mainfrom
research/category-sharing-experiment

Conversation

@Stanback

Copy link
Copy Markdown
Owner

What changes?

Adds a reproducible research-only experiment testing whether category evidence sharing suppresses useful men's/women's differences. It sweeps 81 level/slope sharing combinations and a common-curve control, selects candidates on validation swimmers, and checks the frozen shortlist against the main, later-season and regional holdouts.

The report and aggregate receipt support retaining the current settings: removing sharing raises average error on all three holdouts, while the validation winner offers no consistent improvement. Production calculations and the shipped model remain unchanged; swimmer-level pairs and the fitted-object cache remain local and ignored.

Validation

  • 28 relevant research tests passed (23 evaluator tests and 5 experiment tests).
  • Reconstructed baseline cells exactly match the shipped empirical snapshot; vectorized baseline predictions match scalar predictions on every validation and main-test row.
  • Input hash matches the pinned evaluation receipt; report table matches the aggregate results; git diff --check passed.
  • Application/native suites were not run because this adds standalone offline research without changing consumer or engine behavior.

Review checklist

  • The change is focused and relevant documentation is updated.
  • Validation appropriate to the change is described above.
  • Applicable accessibility, offline, and privacy behavior has been checked.
  • Applicable data attribution, metadata, coverage, and rounding behavior are documented.

@Stanback
Stanback merged commit 645aa14 into main Oct 26, 2026
13 checks passed
@Stanback
Stanback deleted the research/category-sharing-experiment branch October 26, 2026 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant